v Need for establishing benchmarks and metrics for evaluations of blended learning. v What is different about BLENDED that is independent from others. o What is the POLICY for/ rules of blended learning. § No metrics exist to say “this is good, or this is bad” when it comes to the pedagogical methods used by different faculty in reference to blended courses. o What are we trying to accomplish with a blended learning course? § Karen Swan: Integration? How is that measured? § Tom: If you are trying to measure “discussion” and one is done in a classroom and the other is done in the virtual medium, how do we measure the two? § Tanya (UWM): What LEVEL of metrics are we talking about? v There has been a reverse evaluation of blended o Mary: Presentations, article in JLAN: What’s your goal? The extension of that is the measure of success. o If the problem you are trying to solve is “time to degree”, then what is the solution? § How do you blend? And how do you know if you’ve succeeded? § In the grid, a problem was identified and blended was used as the solution. § Now, blended is the given, and needs to go back from there. v Benchmark-based v There is a period of assessment. o Tony (Sloan): What level? Is it something we identify BEFORE and then attempt to measure? § Develop a test ->evaluate -> results speak for themselves = Benchmark o Can you separate out the modality from the overall course? § There’s a core part of the course that is common to any modality · It may be the most significant determinant of the outcome · If the goals that are set for the course, the program, and the institution, and those goals are being met, it then becomes harder to separate out the modality. · We look at “what are the issues pressing higher education over the next 7ish years” · We should identify bigger higher education issues vs. micro issues. v Tanya: Time to degree seems to be the highlighted standard of administrators and institutions due to current economic situation o Consider the audience. § The current state of our economic situation will dictate who is interested in what sort of evaluations should exist. o Karen: What issues are there to which Blended is a solution § Tony agrees v Tony: The faculty starts to assume ownership of the students in higher level courses and professional programs in ways that that they don’t in the GEN EDs (at the program level) o Tanya: When I was an instructor, my colleagues could care less about time to degree. They are concerned with the student’s experience. § What will they take with them? o Tony: They are not exclusive. v Stephen: COST- opportunity cost, as it sits now, is unsustainable. o What does blended do? § It supports brining down the total opportunity cost to successful degree. § The total cost can be driven down by offering a lousy program § You can start there (total economic costs), and decompose from there. (Similar to DuPont model in business) o Part of this would include intake methods. § How many people you retain in degree programs · Failed attempts created. v Tony: Policy Issues- set by state legislatures. Failed open-door policy. o 75% of Higher Ed is in public universities (policy implications) v Patsy: There are multiple audiences o Administrative, Faculty, Student § Defining issues may be better than defining metrics § Will end up at “What percentage online does it have to be blended to be blended” v Karen: Time to degree, persistence, retention, success, post degree employment/graduate school, access o Does this all fold into the economic costs that Steve mentioned v Strategic question- Do we follow the money? (i.e. do what the check-writers want to get the money) o Stimulus packages, grants. v Mary: What about the institutional drivers for going to blended? o Access: the institutional issue is the restriction of access due to certain limitations. § Classrooms, faculty, instructional staff o There are general issues that effect student completion which is the main reasons the institutional drivers are in place v Chuck: Get down a huge laundry list of issues instead of limiting it down. o We tend to gravitate to stuff we CAN measure. § We are really successful when we cant measure anything. v Tony: I am not in the mind to have the Gates foundation moving us in terms of what were doing. v Stephen: Education, next to healthcare, is the second most unsustainable business model there is. o Pricing most of the country out of. § We should have a way to explain how “blended provides value” · If we cant do this, we are suspect of wasting resources. v Mary: Show me where it is cost-effective? o Unless you’re willing to scale the courses, they DO require more resources. § Stephen: Value creation vs. Cost (cheapest education is the one not delivered) · Very hard to evaluate value to society. v Stephen: Sloan should be creating the framework by which institutions can evaluate the progress of blended. v Tony: Are you talking about value-added? More value to those students in risk than to those who aren’t. o Value to who? o Student? Stephen: Society? (wont be on social assistance, wont be involved in crime) v Tanya: Comes back to access. We need to think about pricing people out. o Is a degree even going to matter in 20 years? Do you need 4 years of seat time? o Don’t care about your degree…rather, what can you contribute? v Chuck: 4 models of making money by giving things away. § Unbundling of higher education. Anderson Book. o Stephen: I can see rapid consolidation § Its already going § Why so many programs/disciplines? Why 115 universities in Boston? v Karen: Consolidation in profit and non-profit. v George: Information transfer is not education…crystallizes around blended learning. o The exposure to and application of information is something that happens that non-learning environment, where will it happen? o We cant get stuck on what blended is, but we need to say it’s a good thing. v Karen: We need people who are going to be good citizens. o Having people who are simply competent to do jobs, that may not be the case. v Tanya: Some research says that the education students get and the jobs they will get isn’t even here. o Students are getting an education to solve the world’s problems. o We need to think outside of the box o People who are taking blended learning aren’t taking an 8yr program just to get a Job X, or do Task X. v Tom: Wouldn’t each institution define value depending on their mission? o If your ultimate benchmark is value added to this blended learning process, who do you measure that? v Stephen: Similar to accreditation….What are your goals for doing it? Why are you in blended? Can you even articulate them? Have you thought about them? o Maybe Sloan offers an audit service § Engage in this process § Help them think how the institution gets better v Tony: Sloan not interested in the accrediting/auditing business. o Send a team in. Pure support. Not an audit. § Updating the pillars. o Cost of education keeps going up. § However, more people keep wanting to come to college. v Tanya: Social change o 50 years ago, only the privileged could come to school o Didn’t care about “priced out”, different learning style o Now, different. Everyone should have access to higher ed. o Due to social change, there is an institutional change § Different needs, different markets § Benchmarks should be reflective of the current impact o Students are seen as clients/customers with needs o Often times, pedagogy is forgotten. v Mary: The issue of access keeps coming up. o The pedagogy is an important part of it o Institutional mission is also key § The reason your doing blended learning has to be consistent with the mission, or else it will fail. § The mission needs to change v Chuck: Not be worried about the granularity, but rather identify the constructs that drive this o Identify the guiding pillars to come to a framework for evaluation. v Stephen: Will the pillars stand up? v Karen Swan: I think we want a little smaller grain size. o The pillars didn’t have good metrics to begin with § Saying its as good as F2F didn’t help because I didn’t think F2F was that good to begin with v Tanya: To get there, we have to go bigger before we can come down o Pillars very focused on a certain level of metrics § Need to get into bigger constructs before getting more granular v Stephen: Start with pillars, map the next level down. o Play around with the cost pillar o Even below the sublevel, define each institution for itself. o Real problem: No pillar really gets at the institutional issues o No way at getting consolidation prices with pillars v Stephen: That’s a market force that just happens o Everybody is doing this, maybe we should too. o The pillars aren’t silos. They are systemic. v Mary: A key issue we must be realistic about is the resource/sustainability issue. o Most of the blending doesn’t bring in new dollars like online does. o Blended makes room for new students, but the pipeline exists. v Tanya: Resource pillar instead of cost effectiveness. o Encompasses institutional, societal, unit resources (HR, physical). o Scalability issues. o Scalability and sustainability issues are not independent. v Stephen: Model a new institution with just blended. o The reason blended looks appealing is because they are starving it of resource o It can be more cost consuming. o Retraining employees and retooling them o Bringing in new people wouldn’t really do that. v Tony: Student outcome (value added) gets really complex o CUNY could not afford to sustain o The issueswith the students isn’t just that they aren’t reading, write, arithmetic. o There are social and cultural reasons behind that o Support system v Chuck: Let’s aggregate the models of blended learning to see if they inform in some way. v Tony: Two aspects of the workshop 1. Identify issues (bigger than metrics) 2. Identify models that could possibly resolve those issues. o Forget about the word metrics o Its different in this country. Access has been in US since WWII o The rest of the world only started the idea of access 20 years ago v Stephen: This is giving them a competitive advantage. Very focused approach to learning v Tony: Why do blended? Put the models on the table o Tie them to the higher issues to provide a provocative workshop. o Identify several models. Agrees with Chuck v Mary: That is the one thing people ask for at every conference. We have these models and have done some of presentation. v Stephen: Its been done in pieces, we’ve never pulled it together v Tony: They can identify to College A doing something. We should come up with an overall model that applies to Higher Education in general would be a great service. v Mary: Sent model to CFO- 4 courses that had been F2F and blended. Attempted to capture costs through the elements of the course. Enrollment change. Classroom costs savings. Show a tuition dollar at the bottom o UIC doesn’t know what it costs to deliver a course, maintain classrooms. o Shows that Blended offered more net tuition revenue. o Lots of unknowns. v George: Think about the erosion of public support for public education o Trajectory moving towards 25-50% of state support. o Student want more bang for the buck, cheapen the enterprise v Mary: I think the instructional cost of teaching Blended is higher o More thought o More design into true integration o Maintenance v Tony: Instructional Design: Started thinking about teaching effectively before the technology was here. o Technology sped up the process o How do we move this course to this modality? o F2F is lagging v Mary: If we are looking to compile models and having people be able to use the models, the model should include the driver for that model in the first place. o Aggregation of the models v Chuck: Not thinking about integration-oriented models, but more so theoretical models. v Stephen: Orientation models cast a wider net. v Karen: The community model is the one I like best. o Can you separate institutional/theoretical models? o Commonalities will arise, maybe even inform a question that hasn’t been asked yet. v Patsy: Implementation v. Theoretical? Tanya says both and I agree. (To Chuck) You originally said separate. v Chuck: My immediate interest lies in the theoretical models. Some people would be better at articulating why and how they implemented their Blended learning. v Karen: If we are doing this for people who are thinking of how to apply it to their institution, it should be done at a theoretical level. v Stephen: What would I want answered? o Why do Blended? o How do you do it well? v Tanya: They have a theory in their mind. They probably know why they do blended. o There is something they are trying to overcome. o Its going be one of these issues and the models will provide the solution v Chuck: “Yeah, but you didn’t tell me how to do it.” v Karen: A general idea of how it can be implemented. o What you want is an analytical model o Something that bridges the jump from why to how? o Slow them down and show connections between purpose and actual gain. o Models? o Ecosystem model: how EVERYTHING is put together. (Stephen and Mary) v George: Reasons to do blended learning could be tied to the pillars. o Put problems into categories and under those, address how your institution addresses those. o Faculty satisfaction- faculty cant make the jump at once o Student satisfaction- students aren’t regulated enough to be applied. v Tanya: Macro issues are often looked over. Providing the bridge is key. v Mary: Systemic approach. o It can’t be successful if its simply at a course level o All goals and issues aren’t addressed. v Stephen: Can’t be just pedagogically focused (need student support service) o Can’t be successful just based on technology o Can’t be successful just to make money o There must be this ecosystem which is allowed to flourish. v Tony: Collect case studies of failed programs o Good to highlight the failures. o Summary of Outcomes Next Steps: v Tony: We set up a workshop in Oct/Nov. Discuss the macro issues and provide conceptual and practical models for addressing these issues v Chuck: I will volunteer the UCF group to begin finding as many theoretical models as can be found and begin circulating them to the group. Collage-based v Tony: I think we need to get a list of big issues. I will need help. We could probably pull from EduCause??? (Sept 15). Critical Issues for Blended Learning. v Mary: I will contribute to the Wiki on things to consider when doing blended. v Mary: I will circulate the case study template. UIC will also be responsible for meeting coordination. Resurrect grid that was done with George. o Once a month conference call. v Stephen: We have an opportunity to frame the conversation. Get there quickly before someone else makes the argument for us. To bring this to a chancellor, it needs to be packaged. v Tanya: Set up Wiki to consolidate brainstorming of issues. Move to…
Identify Issues
Models (conceptual/applied)
Use workshop to test out ideas. If feedback is good, move to writing a paper.
Get support from powers that be (presidents, chancellors, etc)
Must be packaged to do this.
End outcome is to send out something to the policy makers.
They are already legislating. Game is already starting.
Accrediting bodies are changing the definitions of what online/blended is.
Retraining a population- life long learning
Blended/online learning is good for that end
This is another aspect of the “value added” conversation.
Go after the alumni
Support them through blended learning by continuous training
April 19, 2010
v Need for establishing benchmarks and metrics for evaluations of blended learning.
v What is different about BLENDED that is independent from others.
o What is the POLICY for/ rules of blended learning.
§ No metrics exist to say “this is good, or this is bad” when it comes to the pedagogical methods used by different faculty in reference to blended courses.
o What are we trying to accomplish with a blended learning course?
§ Karen Swan: Integration? How is that measured?
§ Tom: If you are trying to measure “discussion” and one is done in a classroom and the other is done in the virtual medium, how do we measure the two?
§ Tanya (UWM): What LEVEL of metrics are we talking about?
v There has been a reverse evaluation of blended
o Mary: Presentations, article in JLAN: What’s your goal? The extension of that is the measure of success.
o If the problem you are trying to solve is “time to degree”, then what is the solution?
§ How do you blend? And how do you know if you’ve succeeded?
§ In the grid, a problem was identified and blended was used as the solution.
§ Now, blended is the given, and needs to go back from there.
v Benchmark-based
v There is a period of assessment.
o Tony (Sloan): What level? Is it something we identify BEFORE and then attempt to measure?
§ Develop a test ->evaluate -> results speak for themselves = Benchmark
o Can you separate out the modality from the overall course?
§ There’s a core part of the course that is common to any modality
· It may be the most significant determinant of the outcome
· If the goals that are set for the course, the program, and the institution, and those goals are being met, it then becomes harder to separate out the modality.
· We look at “what are the issues pressing higher education over the next 7ish years”
· We should identify bigger higher education issues vs. micro issues.
v Tanya: Time to degree seems to be the highlighted standard of administrators and institutions due to current economic situation
o Consider the audience.
§ The current state of our economic situation will dictate who is interested in what sort of evaluations should exist.
o Karen: What issues are there to which Blended is a solution
§ Tony agrees
v Tony: The faculty starts to assume ownership of the students in higher level courses and professional programs in ways that that they don’t in the GEN EDs (at the program level)
o Tanya: When I was an instructor, my colleagues could care less about time to degree. They are concerned with the student’s experience.
§ What will they take with them?
o Tony: They are not exclusive.
v Stephen: COST- opportunity cost, as it sits now, is unsustainable.
o What does blended do?
§ It supports brining down the total opportunity cost to successful degree.
§ The total cost can be driven down by offering a lousy program
§ You can start there (total economic costs), and decompose from there. (Similar to DuPont model in business)
o Part of this would include intake methods.
§ How many people you retain in degree programs
· Failed attempts created.
v Tony: Policy Issues- set by state legislatures. Failed open-door policy.
o 75% of Higher Ed is in public universities (policy implications)
v Patsy: There are multiple audiences
o Administrative, Faculty, Student
§ Defining issues may be better than defining metrics
§ Will end up at “What percentage online does it have to be blended to be blended”
v Karen: Time to degree, persistence, retention, success, post degree employment/graduate school, access
o Does this all fold into the economic costs that Steve mentioned
v Strategic question- Do we follow the money? (i.e. do what the check-writers want to get the money)
o Stimulus packages, grants.
v Mary: What about the institutional drivers for going to blended?
o Access: the institutional issue is the restriction of access due to certain limitations.
§ Classrooms, faculty, instructional staff
o There are general issues that effect student completion which is the main reasons the institutional drivers are in place
v Chuck: Get down a huge laundry list of issues instead of limiting it down.
o We tend to gravitate to stuff we CAN measure.
§ We are really successful when we cant measure anything.
v Tony: I am not in the mind to have the Gates foundation moving us in terms of what were doing.
v Stephen: Education, next to healthcare, is the second most unsustainable business model there is.
o Pricing most of the country out of.
§ We should have a way to explain how “blended provides value”
· If we cant do this, we are suspect of wasting resources.
v Mary: Show me where it is cost-effective?
o Unless you’re willing to scale the courses, they DO require more resources.
§ Stephen: Value creation vs. Cost (cheapest education is the one not delivered)
· Very hard to evaluate value to society.
v Stephen: Sloan should be creating the framework by which institutions can evaluate the progress of blended.
v Tony: Are you talking about value-added? More value to those students in risk than to those who aren’t.
o Value to who?
o Student? Stephen: Society? (wont be on social assistance, wont be involved in crime)
v Tanya: Comes back to access. We need to think about pricing people out.
o Is a degree even going to matter in 20 years? Do you need 4 years of seat time?
o Don’t care about your degree…rather, what can you contribute?
v Chuck: 4 models of making money by giving things away.
§ Unbundling of higher education. Anderson Book.
o Stephen: I can see rapid consolidation
§ Its already going
§ Why so many programs/disciplines? Why 115 universities in Boston?
v Karen: Consolidation in profit and non-profit.
v George: Information transfer is not education…crystallizes around blended learning.
o The exposure to and application of information is something that happens that non-learning environment, where will it happen?
o We cant get stuck on what blended is, but we need to say it’s a good thing.
v Karen: We need people who are going to be good citizens.
o Having people who are simply competent to do jobs, that may not be the case.
v Tanya: Some research says that the education students get and the jobs they will get isn’t even here.
o Students are getting an education to solve the world’s problems.
o We need to think outside of the box
o People who are taking blended learning aren’t taking an 8yr program just to get a Job X, or do Task X.
v Tom: Wouldn’t each institution define value depending on their mission?
o If your ultimate benchmark is value added to this blended learning process, who do you measure that?
v Stephen: Similar to accreditation….What are your goals for doing it? Why are you in blended? Can you even articulate them? Have you thought about them?
o Maybe Sloan offers an audit service
§ Engage in this process
§ Help them think how the institution gets better
v Tony: Sloan not interested in the accrediting/auditing business.
o Send a team in. Pure support. Not an audit.
§ Updating the pillars.
o Cost of education keeps going up.
§ However, more people keep wanting to come to college.
v Tanya: Social change
o 50 years ago, only the privileged could come to school
o Didn’t care about “priced out”, different learning style
o Now, different. Everyone should have access to higher ed.
o Due to social change, there is an institutional change
§ Different needs, different markets
§ Benchmarks should be reflective of the current impact
o Students are seen as clients/customers with needs
o Often times, pedagogy is forgotten.
v Mary: The issue of access keeps coming up.
o The pedagogy is an important part of it
o Institutional mission is also key
§ The reason your doing blended learning has to be consistent with the mission, or else it will fail.
§ The mission needs to change
v Chuck: Not be worried about the granularity, but rather identify the constructs that drive this
o Identify the guiding pillars to come to a framework for evaluation.
v Stephen: Will the pillars stand up?
v Karen Swan: I think we want a little smaller grain size.
o The pillars didn’t have good metrics to begin with
§ Saying its as good as F2F didn’t help because I didn’t think F2F was that good to begin with
v Tanya: To get there, we have to go bigger before we can come down
o Pillars very focused on a certain level of metrics
§ Need to get into bigger constructs before getting more granular
v Stephen: Start with pillars, map the next level down.
o Play around with the cost pillar
o Even below the sublevel, define each institution for itself.
o Real problem: No pillar really gets at the institutional issues
o No way at getting consolidation prices with pillars
v Stephen: That’s a market force that just happens
o Everybody is doing this, maybe we should too.
o The pillars aren’t silos. They are systemic.
v Mary: A key issue we must be realistic about is the resource/sustainability issue.
o Most of the blending doesn’t bring in new dollars like online does.
o Blended makes room for new students, but the pipeline exists.
v Tanya: Resource pillar instead of cost effectiveness.
o Encompasses institutional, societal, unit resources (HR, physical).
o Scalability issues.
o Scalability and sustainability issues are not independent.
v Stephen: Model a new institution with just blended.
o The reason blended looks appealing is because they are starving it of resource
o It can be more cost consuming.
o Retraining employees and retooling them
o Bringing in new people wouldn’t really do that.
v Tony: Student outcome (value added) gets really complex
o CUNY could not afford to sustain
o The issueswith the students isn’t just that they aren’t reading, write, arithmetic.
o There are social and cultural reasons behind that
o Support system
v Chuck: Let’s aggregate the models of blended learning to see if they inform in some way.
v Tony: Two aspects of the workshop
1. Identify issues (bigger than metrics)
2. Identify models that could possibly resolve those issues.
o Forget about the word metrics
o Its different in this country. Access has been in US since WWII
o The rest of the world only started the idea of access 20 years ago
v Stephen: This is giving them a competitive advantage. Very focused approach to learning
v Tony: Why do blended? Put the models on the table
o Tie them to the higher issues to provide a provocative workshop.
o Identify several models. Agrees with Chuck
v Mary: That is the one thing people ask for at every conference. We have these models and have done some of presentation.
v Stephen: Its been done in pieces, we’ve never pulled it together
v Tony: They can identify to College A doing something. We should come up with an overall model that applies to Higher Education in general would be a great service.
v Mary: Sent model to CFO- 4 courses that had been F2F and blended. Attempted to capture costs through the elements of the course. Enrollment change. Classroom costs savings. Show a tuition dollar at the bottom
o UIC doesn’t know what it costs to deliver a course, maintain classrooms.
o Shows that Blended offered more net tuition revenue.
o Lots of unknowns.
v George: Think about the erosion of public support for public education
o Trajectory moving towards 25-50% of state support.
o Student want more bang for the buck, cheapen the enterprise
v Mary: I think the instructional cost of teaching Blended is higher
o More thought
o More design into true integration
o Maintenance
v Tony: Instructional Design: Started thinking about teaching effectively before the technology was here.
o Technology sped up the process
o How do we move this course to this modality?
o F2F is lagging
v Mary: If we are looking to compile models and having people be able to use the models, the model should include the driver for that model in the first place.
o Aggregation of the models
v Chuck: Not thinking about integration-oriented models, but more so theoretical models.
v Stephen: Orientation models cast a wider net.
v Karen: The community model is the one I like best.
o Can you separate institutional/theoretical models?
o Commonalities will arise, maybe even inform a question that hasn’t been asked yet.
v Patsy: Implementation v. Theoretical? Tanya says both and I agree. (To Chuck) You originally said separate.
v Chuck: My immediate interest lies in the theoretical models. Some people would be better at articulating why and how they implemented their Blended learning.
v Karen: If we are doing this for people who are thinking of how to apply it to their institution, it should be done at a theoretical level.
v Stephen: What would I want answered?
o Why do Blended?
o How do you do it well?
v Tanya: They have a theory in their mind. They probably know why they do blended.
o There is something they are trying to overcome.
o Its going be one of these issues and the models will provide the solution
v Chuck: “Yeah, but you didn’t tell me how to do it.”
v Karen: A general idea of how it can be implemented.
o What you want is an analytical model
o Something that bridges the jump from why to how?
o Slow them down and show connections between purpose and actual gain.
o Models?
o Ecosystem model: how EVERYTHING is put together. (Stephen and Mary)
v George: Reasons to do blended learning could be tied to the pillars.
o Put problems into categories and under those, address how your institution addresses those.
o Faculty satisfaction- faculty cant make the jump at once
o Student satisfaction- students aren’t regulated enough to be applied.
v Tanya: Macro issues are often looked over. Providing the bridge is key.
v Mary: Systemic approach.
o It can’t be successful if its simply at a course level
o All goals and issues aren’t addressed.
v Stephen: Can’t be just pedagogically focused (need student support service)
o Can’t be successful just based on technology
o Can’t be successful just to make money
o There must be this ecosystem which is allowed to flourish.
v Tony: Collect case studies of failed programs
o Good to highlight the failures.
o Summary of Outcomes
Next Steps:
v Tony: We set up a workshop in Oct/Nov. Discuss the macro issues and provide conceptual and practical models for addressing these issues
v Chuck: I will volunteer the UCF group to begin finding as many theoretical models as can be found and begin circulating them to the group. Collage-based
v Tony: I think we need to get a list of big issues. I will need help. We could probably pull from EduCause??? (Sept 15). Critical Issues for Blended Learning.
v Mary: I will contribute to the Wiki on things to consider when doing blended.
v Mary: I will circulate the case study template. UIC will also be responsible for meeting coordination. Resurrect grid that was done with George.
o Once a month conference call.
v Stephen: We have an opportunity to frame the conversation. Get there quickly before someone else makes the argument for us. To bring this to a chancellor, it needs to be packaged.
v Tanya: Set up Wiki to consolidate brainstorming of issues.
Move to…